SIMULATION  MODELING  OF  ADVANCED  PILOT  TRAINING:  THE 
EFFECTS  OF  A  NEW  AIRCRAFT  FAMILY  OF  SYSTEMS 


THESIS 


Bryngel  J.  Erickson,  Captain,  USAF 
AFIT-ENS-14-M-05 

DEPARTMENT  OF  THE  AIR  FORCE 
AIR  UNIVERSITY 

AIR  FORCE  INSTITUTE  OF  TECHNOLOGY 
Wright-Patterson  Air  Force  Base,  Ohio 


DISTRIBUTION  STATEMENT  A. 

APPROVED  FOR  PUBLIC  RELEASE;  DISTRIBUTION  UNLIMITED. 


The  views  expressed  in  this  thesis  are  those  of  the  author  and  do  not  reflect  the  official 
policy  or  position  of  the  United  States  Air  Force,  Department  of  Defense,  or  the  United 
States  Government.  This  material  is  declared  a  work  of  the  U.S.  Government  and  is  not 
subject  to  copyright  protection  in  the  United  States. 


AFIT-ENS-14-M-05 


SIMULATION  MODELING  OF  ADVANCED  PILOT  TRAINING:  THE 
EFFECTS  OF  A  NEW  AIRCRAFT  FAMILY  OF  SYSTEMS 

THESIS 

Presented  to  the  Faculty 
Department  of  Operational  Sciences 
Graduate  School  of  Engineering  and  Management 
Air  Force  Institute  of  Technology 
Air  University 

Air  Education  and  Training  Command 
In  Partial  Fulfillment  of  the  Requirements  for  the 
Degree  of  Master  of  Science  in  Logistics  and  Supply  Chain  Management 


Bryngel  J.  Erickson,  BS 
Captain,  USAF 


March  2014 

DISTRIBUTION  STATEMENT  A. 

APPROVED  FOR  PUBLIC  RELEASE;  DISTRIBUTION  UNLIMITED. 


AFIT-ENS-14-M-05 


SIMULATION  MODELING  OF  ADVANCED  PILOT  TRAINING:  THE 
EFFECTS  OF  A  NEW  AIRCRAFT  FAMILY  OF  SYSTEMS 

Bryngel  J.  Erickson,  BS 
Captain,  USAF 


Approved: 

17-03-14 
Date 


_ //signed// _ 

Darryl  K.  Ahner,  PhD  (Chainnan) 


_ //signed// _ 

J.  O.  Miller,  PhD  (Member) 


17-03-14 

Date 


AFIT-ENS-14-M-05 


Abstract 

This  research  develops  a  tool  to  aid  Air  Force  decision  makers  in  the  acquisition 
of  a  new  trainer  aircraft  family  of  systems.  The  study  utilizes  discrete  event  simulation  to 
model  the  effects  a  prospective  aircraft  family  of  systems  has  on  the  flow  of  student 
pilots  through  training  processes.  Previous  research  has  produced  models  for  student 
throughput  or  aircraft  availability,  this  research  focuses  on  the  intersection  of  both. 
Analysis  of  results  provides  insights  into  the  quantity  of  resources  required  at  the 
differing  levels  of  performance  to  sustain  a  desired  throughput  of  pilot  graduates. 
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SIMULATION  MODELING  OF  ADVANCED  PILOT  TRAINING:  THE 


EFFECTS  OF  A  NEW  AIRCRAFT  FAMILY  OF  SYSTEMS 


I.  Introduction 


Background 

The  United  States  Air  Force  maintains  a  rigorous  pilot  training  program  to  ensure 
pilots  are  prepared  to  meet  future  mission  requirements.  Students  preparing  to  work  with 
lighter  or  bomber  aircraft  train  with  the  T-38  Talon  during  Advanced  Pilot  Training.  The 
training  is  a  combination  of  classroom  instruction,  computer  based  training,  ground  based 
trainer  system  (GBTS)  exercises,  and  aircraft  flying  exercises.  The  intent  of  the  program 
is  to  provide  students  with  all  the  basic  skills  necessary  to  transition  to  their  follow-on 
aircraft. 

The  T-38  has  been  an  effective  training  tool  in  the  past;  however,  it  is  becoming  a 
less  desirable  trainer  aircraft  as  time  progresses.  Originally  developed  as  a  lead  in  trainer 
for  the  F-4  Phantom,  the  T-38  now  trains  fourth  and  fifth  generation  aircraft  pilots 
(Trimble,  2011).  The  definitions  for  aircraft  generation  characteristics  vary  by  industries 
and  nations.  In  general  however,  the  T-38  and  F-4  are  third  generation  aircraft  while 
more  advanced  systems  like  the  F-15  and  F-16  are  fourth  generation  aircraft  (Pike,  2012). 
Development  of  fifth  generation  fighters  like  the  F-22  and  F-35  has  created  training 
requirements  that  the  third  generation  trainer  is  unequipped  to  handle  (Trimble,  2011). 

To  compensate  for  deficiencies  in  training,  the  Air  Force  sends  graduates  to  additional 
training  with  F-16  aircraft  to  bridge  the  gaps  in  training  prior  to  a  fifth  generation  fighter 
assignment. 
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In  addition  to  technological  shortcomings,  the  T-38  fleet  is  becoming  increasingly 
more  expensive  to  maintain.  The  aircraft  are  old,  hailing  from  a  fleet  built  for  the  Air 
Force  between  1961  and  1972  (Factsheet,  2005:1).  Although  each  aircraft  was  designed 
to  last  7,000  flight  hours,  the  average  aircraft  has  flown  well  over  15,000  flight  hours 
(Trimble,  2011).  A  combination  of  metal  fatigue,  parts  availability,  corrosion,  fuel 
consumption,  and  other  factors  create  costs  that  the  Air  Force  projects  to  climb 
significantly  in  the  coming  years.  In  order  to  avoid  these  costs  and  improve  the  quality  of 
pilot  training,  the  Air  Force  has  decided  to  pursue  a  replacement  aircraft  family  of 
systems  (FoS)  for  the  T-38.  A  family  of  systems  includes  not  only  the  aircraft,  but  all  the 
simulators,  computers,  and  training  materials  required  to  conduct  pilot  training. 

Acquisition  Approach 

One  approach  to  selecting  a  replacement  aircraft  FoS  is  a  multistage  vetting 
process  that  considers  both  cost  and  non-cost  components  as  depicted  in  Figure  1.  In  the 
initial  phase,  the  vendor’s  proposal  would  be  required  to  meet  specific  requirements  set 
by  the  Air  Force.  Any  proposal  that  fails  to  meet  the  baseline  requirements  disqualifies 
itself  from  further  consideration.  The  next  phase  of  competition  involves  adjusting  the 
total  proposed  price  by  considering  costs  not  covered  in  the  initial  purchase  price.  These 
costs  come  from  a  variety  of  FoS  attributes  such  as  aircraft  fuel  efficiency,  additional 
construction  requirements,  operation  and  support  requirements,  as  well  as  others. 

Decision  makers  must  choose  attributes  that  are  important  to  them  and  are  quantifiable  in 
dollar  figures.  After,  the  adjusted  price  is  calculated,  the  non-mandatory  requirements 
are  compared  and  scored  to  give  credit  to  vendor  proposals  that  outperform  or  penalize 
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proposals  that  underperform  in  a  given  criteria.  The  combination  of  adjusted  price  and 
additional  scoring  detennines  which  aircraft  FoS  the  U.S.  Air  Force  will  select.  This 
thesis  will  focus  on  the  multivariable  price  adjustment  portion  of  the  procurement 
strategy. 


Figure  1.  T-X  Procurement  Strategy 


Pilot  Training 

Air  Force  pilot  training  is  a  multiphase  process  involving  several  bases  and 
courses  of  instruction.  Pilot  trainees  all  begin  taking  the  same  courses  that  focus  on 
general  aviation  principles  and  skills.  Students  then  separate  into  specialized  tracks  in 
Advanced  Pilot  Training  (APT)  for  functionally  emphasized  training  such  as 
airlift/tanker,  fighter/bomber,  turboprop,  and  helicopter.  Following  APT,  students 
transition  into  airframe  specific  training,  such  as  F-22  or  C-17,  to  prepare  for  an 
operational  unit  assignment. 
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Figure  2.  Advanced  Pilot  Training 


The  shaded  portion  of  Figure  2  depicts  T-38s  utilized  by  the  Air  Education  and 
Training  Command,  primarily  for  the  advanced  portion  of  training  tailored  to  future 
fighter  and  bomber  pilots.  The  throughput  of  pilots  depends  on  the  skills  of  individual 
students,  proper  timing  and  delivery  of  course  materials,  as  well  as  the  availability  of 
resources.  There  are  syllabus  optimization  studies  and  sortie  generation  studies  that 
focus  on  various  aspects  of  these  events.  The  simulation  developed  in  this  thesis  will 
focus  on  the  crossroads  of  syllabus  requirements  and  aircraft  and  simulator  availability. 

Problem  Statement 

Air  Force  decision  makers  need  a  tool  to  assist  in  evaluating  costs  associated  with 
a  proposed  aircraft  FoS.  The  tool  needs  to  be  able  to  distinguish  between  aircraft  that 
perfonn  at  different  levels  of  logistics  and  maintenance  capabilities.  The  tool  also  needs 
to  provide  insight  into  how  future  costs  may  be  impacted  by  a  FoS’s  level  of 
perfonnance. 

Research  Objectives 

This  research  will  deliver  a  model  that  uses  Arena®  based  simulation  to  represent 
pilot  training  syllabi.  Since  the  current  syllabi  do  not  include  lessons  for  all  required 
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training  that  the  proposed  FoS  will  be  able  to  accomplish,  the  model  architecture  will 
provide  a  template  to  support  future  adjustments  in  course  work.  Decision  makers  will 
use  the  model  to  gain  insight  on  how  the  proposed  aircraft  FoS  could  affect  mission 
accomplishment.  In  this  case,  student  pilot  throughput  detennines  accomplishment  of  the 
mission.  Using  this  simulation,  decision  makers  will  be  able  to  project  how  many  aircraft 
and  ground  based  trainer  systems  will  be  required  to  maintain  a  given  level  of  student 
throughput.  In  addition,  the  model  will  show  the  sensitivity  of  student  throughput  to 
variation  in  FoS  performance.  The  number  of  aircraft  and  GBTS  needed  to  produce  a 
desired  quantity  of  students  will  then  be  used  to  adjust  the  total  cost. 

Scope 

This  research  uses  course  flow  from  the  T-38C  Specialized  Undergraduate  Pilot 
Training  and  USAF  Introduction  to  Fighter  Fundamentals  syllabi  to  model  student 
progress.  No  adjustments  to  course  flow  are  made  for  optimization  purposes.  Base  and 
squadron  level  requirements  for  aircraft  and  GBTS  fonn  the  basis  for  fleet  requirements. 
The  simulation  does  not  factor  in  depot  repair  requirements  but  leaves  those  adjustments 
for  calculation  after  the  fleet  size  is  detennined. 
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II.  Literature  Review 


Introduction 

This  chapter  describes  some  of  the  background  knowledge  used  to  develop  this 
study.  The  chapter  outlines  benefits  of  simulation  studies  as  well  as  the  reasons  for 
utilizing  Arena®  software.  Applicable  simulation  studies  are  reviewed  and  simulation 
models  are  discussed. 

Simulations 

Process  flows  like  the  ones  represented  in  the  pilot  training  syllabi  can  be 
represented  in  a  number  ways,  to  include  spreadsheet,  simple  analytic  or  mathematical 
approaches.  Although  other  methods  could  be  used,  Carson  outlines  several  good 
reasons  why  a  simulation  approach  is  preferable  for  this  study  (Carson,  2005).  When 
various  components  interact  or  function  interdependently,  the  system  becomes  very 
complex  and  difficult  to  predict  the  effects  of  altering  one  component  (Carson,  2005:17). 
Since  pilot  training  is  a  combination  of  multiple  moving  parts,  it  is  easier  to  let  the 
simulation  play  out  scenarios  than  to  directly  calculate  every  interaction.  Simulation  is 
also  preferable  when  developing  a  new  system  or  experimenting  with  new  or  different 
demand  (Carson,  2005:17).  The  structure  of  this  simulation  not  only  lets  decision  makers 
alter  the  frequency  and  size  of  student  pilot  classes,  it  also  provides  a  template  for  future 
changes  in  coursework  and  their  impact  on  needed  resources.  Simulations  are  also 
desirable  when  a  large  financial  investment  is  involved,  the  system  is  not  fully 
understood,  or  considerable  risk  is  present  (Carson,  2005: 17).  Developing  a  new  aircraft 
is  full  of  high  cost  risk  with  future  outcomes  not  usually  certain.  This  model  will  help 
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decision  makers  see  the  impacts  that  may  occur  if  they  choose  an  aircraft  with 
perfonnance  level  x  over  an  aircraft  with  performance  level  y.  Perhaps  the  model  will 
show  that  the  impact  is  negligible  and  extra  money  should  not  be  spent  on  the  higher 
perfonning  system.  Another  reason  for  using  simulation  is  for  situations  where  decision 
makers  must  agree  upon  multiple  assumptions  (Carson,  2005: 17).  Through  this  model, 
the  assumptions  about  the  system  and  their  effects  can  be  clearly  seen.  More  importantly, 
if  decision  makers  do  not  agree  on  baseline  assumptions,  they  can  easily  alter  the  model 
and  examine  the  results.  The  flexibility  to  change  assumptions  in  the  model  saves 
decision  makers  from  having  to  rebuild  a  completely  new  analysis  from  scratch  each  time 
an  assumption  becomes  invalid. 

To  develop  a  simulation  study,  choosing  the  proper  simulation  platform  is  very 
important.  This  simulation  utilizes  Arena®  application  software  to  build  the  syllabi 
model.  Arena®  is  a  practical  choice  for  this  research  because  of  accessibility  and 
affordability  to  the  Air  Force.  Arena®  models  have  been  used  extensively  in  the  past  so 
operating  license  access  and  familiarity  with  the  program  will  not  be  a  large  obstacle  for 
decision  makers.  The  software  is  also  a  simple-to-use  graphical  module  interface  that  lets 
novice  programmers  quickly  understand  and  manipulate  complex  logical  structures.  The 
graphical  modules  are  ideal  for  processes  flow  analysis  because  they  allow  reviewers  to 
see  how  entities  move  and  interact  through  the  model.  The  overall  effectiveness  and 
simplicity  of  Arena®  make  the  software  an  ideal  choice. 
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Previous  Studies 


Several  studies  have  looked  at  resource  constraints  on  pilot  training  programs. 
Although  the  focus  was  different  in  each  study,  two  previous  research  projects  were 
particularly  applicable  to  this  thesis  project. 

First  Lieutenant  Okal  developed  an  AFIT  study  in  2008  that  looked  at  the  training 
process  of  fighter  pilots  in  the  Turkish  Air  Force.  The  study  analyzed  one  course  of 
instruction  broken  into  multiphase  successive  modules.  Okal  developed  an  Arena® 
based  simulation  model  to  analyze  affects  on  total  training  time  that  would  occur  with 
varying  levels  of  resource  availability.  As  the  number  of  instructor  pilots,  student  pilots, 
and  aircraft  changed,  the  model  calculated  the  total  time  required  for  a  student  to 
complete  the  required  training  (Okal,  2008).  Analysis  of  the  model  provided  decision 
makers  with  insight  into  how  many  of  which  resources  would  be  required  to  support 
future  student  class  sizes  (Okal,  2008:67-70).  A  limitation  of  Okal’s  study  was  the  lack 
of  maintenance  and  logistics  variability.  Once  set,  the  number  or  aircraft  available  for 
student  use  did  not  vary  so  the  effect  of  maintenance  delays  was  lost  in  the  system. 

Raivio  et  al.  developed  an  Arena®  based  simulation  that  modeled  the  use  of  the 
Bae  Hawk  Mk5 1  aircraft.  In  this  model,  the  Finnish  Air  Force’s  pilot  training  program 
forms  the  framework  of  daily  operations  (Raivio  et  al.,  2001 : 190).  The  model  uses  daily 
training  sorties  to  calculate  aircraft  flying  hours  as  a  metric  for  maintenance 
requirements.  After  each  aircraft  lands,  the  model  assess  probability  of  breaks  based  on 
historical  data  to  determine  one  of  three  levels  of  possible  maintenance.  The  model  is 
able  to  use  very  specific  distributions  for  each  level  of  maintenance  repair  time  due  to  an 
abundance  of  statistical  data.  From  the  detailed  simulation  of  break  and  fix  rates,  the 
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simulation  team  developed  a  four  year  projection  of  daily  aircraft  availability  rates.  The 
team  was  also  able  to  show  how  the  rates  would  change  if  manpower  capability  was 
scaled  up  or  down  (Raivio  et  ah,  2001 : 194). 

LCOM 

The  Logistics  Composite  Model  (LCOM)  was  designed  by  the  Air  Force  and  The 
Rand  Corporation  in  the  1960s  as  a  sortie  generation  analysis  tool  (Boyle,  1990).  Figure 
3  below  shows  a  very  basic  format  of  the  LCOM  functionality. 


In  the  LCOM  model,  aircraft  are  the  flow-units  that  process  through  the  model. 
The  simulation  moves  the  aircraft  through  phases  of  preflight,  flying,  and  post  flight 
activities  to  simulate  the  work  of  a  maintenance  organization  (Boyle,  1990).  The  model’s 
modules  pull  resources  required  for  task  completion  and  hold  them  for  the  specified 
required  time  to  complete  the  task.  If  there  are  not  enough  resources  for  a  specific 
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operation,  the  aircraft  waits  until  resources  are  available.  If  too  many  backlogs  occur, 
flying  missions  are  canceled  to  reduce  the  demand  on  aircraft.  The  user  inputs 
information  about  the  number  of  available  resources  to  pull  from.  The  user  also  defines 
how  long  events  are  expected  to  take  and  what  their  statistical  distributions  look  like.  By 
adjusting  the  levels  of  resources  and  altering  the  perfonnance  of  aircraft,  a  decision 
maker  can  get  a  better  idea  of  what  he/she  will  need  to  support  a  certain  level  of  aircraft 
activity.  Decision  makers  can  use  the  tool  for  a  variety  of  applications  but  the  focus  is  to 
detennine  appropriate  levels  of  resources,  including  manpower,  to  meet  a  flying  demand 
(Boyle,  1990). 

One  of  the  downsides  of  the  LOOM  model  is  it  is  very  large  and  complicated  to 
handle.  Significant  time  is  required  to  prepare  input  data  to  feed  into  LOOM  and 
additional  time  is  required  to  make  sense  of  over  200  output  statistics  generated  (ACC, 
2013).  In  order  to  be  able  to  use  the  tool  effectively,  the  user  would  require  several 
months  of  full  time  study  and  the  aid  of  technical  support  (ACC,  2013). 

SIMFORCE 

The  Scalable  Integration  Model  for  Objective  Resource  Capability  Evaluations 
(SIMFORCE)  is  a  simulation  model  developed  under  the  Air  Force  Research  Lab  as  a 
simple  tool  to  provide  logisticians  an  ability  to  measure  the  effects  of  logistics  constraints 
on  operational  capabilities  (Brown  and  Powers,  2000: 1050).  SIMFORCE  aims  to  reduce 
wing  level  analysis  gained  from  programs  like  LCOM  down  to  a  unit  level  while  also 
including  more  detailed  infonnation  about  the  types  of  resources  required  (Brown  and 
Powers,  2000:1050).  The  format  of  this  model  allows  unit  level  decision  makers  to 
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conduct  desktop  analysis  without  the  need  for  a  large  database  or  technical  support.  The 
program  quickly  shows  the  impact  on  sortie  generation  if  assets  become  unavailable. 

Another  feature  of  SIMFORCE  is  inputs  are  accepted  and  outputs  generated 
without  the  user  having  to  look  at  any  of  the  process  (Brown  and  Powers,  2000: 1050). 
The  black  box  approach  lets  users  interface  with  simple  graphical  interface  queues  and 
excel  forms  and  does  not  require  them  to  have  skills  in  programming.  Not  only  does  the 
layered  approach  simplify  functions  for  the  user,  it  can  also  shield  the  program  from  view 
so  that  logic  is  not  accidently  altered  or  purposefully  gamed. 

Summary 

The  research  and  programs  cited  in  this  chapter  show  that  aircraft  generation  and 
student  flow  studies  can  be  approach  from  a  variety  of  ways.  An  Arena®  simulation  will 
provide  simple  to  understand  analysis  capabilities  that  can  be  delivered  to  decision 
makers  in  a  number  of  different  formats.  An  open  program  may  be  best  for  some 
decision  makers  while  a  black  box  approach  may  be  better  for  others. 

The  model  developed  for  this  study  will  not  simply  focus  on  student  flying  tasks 
or  daily  sortie  generation  capabilities.  This  model  will  be  a  synthesis  of  past  work  that 
shows  how  student  progress  is  affected  by  the  varying  availability  of  GBTS  and  aircraft 
at  different  times  of  the  day. 
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III.  Methodology 


Chapter  Overview 

The  model  developed  for  this  study  is  designed  to  simulate  how  the  number  of 
aircraft  and  ground  based  trainer  systems  (GBTS),  as  well  as  their  level  of  performance, 
affects  the  flow  of  students  through  pilot  training.  To  measure  the  training  program’s 
effectiveness,  the  simulation  reports  the  number  of  students  that  complete  the  training  as 
well  as  the  number  of  students  that  are  delayed  throughout  the  training  due  to  a  lack  of 
available  resources.  The  user  adjusts  the  quantity  of  aircraft  and  GBTS  to  identify  the 
minimum  number  of  aircraft  and  GBTS  required  for  consistent  student  throughput.  In 
addition,  the  user  adjusts  aircraft  and  GBTS  performance  levels  to  affect  the  availability 
of  these  resources  to  support  student  flow  through  the  model.  This  chapter  outlines  the 
functionality  of  the  model  starting  from  a  course  view  and  narrowing  down  to  a  daily 
process  and  event  focused  perspective. 

Assumptions  and  limitations 

The  statistician  George  Box  noted,  “All  models  are  wrong,  but  some  are  useful.” 
The  model  in  this  study  is  no  exception  to  Box’s  remark.  Multiple  assumptions  and 
limitations  reside  in  this  study  in  order  to  create  a  product  that  can  provide  insights  to 
decision  makers  even  though  the  model  does  not  exactly  replicate  reality. 

•  The  performance  level  of  the  resources  assume  that  20,000  fleet  hours 
have  already  occurred.  The  model  does  not  reduce  resource  performance 
as  flying  hours  increase. 
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•  The  model  excludes  federal  holidays  and  Christmas  break  from  the 
training  year  to  better  capture  class  overlap  durations.  The  training  year 
consists  of  343  training  days  and  weekends. 

•  Each  class  will  have  the  same  number  of  training  days  to  complete  the 
respective  tracks.  The  interval  between  class  start  dates  is  detennined  by 
the  interval  between  scheduled  graduation  dates. 

•  Students  progress  through  the  model  as  outlined  by  the  syllabi.  The  model 
segments  the  training  days  into  five  training  day  weeks  separated  by 
weekends.  Within  a  week,  a  student  cannot  progress  more  than  one  day 
ahead  of  schedule.  Delays  due  to  no  available  resources  may  be  made  up 
within  the  week.  If  the  student  does  not  catch  up  by  the  weekend,  the 
model  holds  them  back  a  week. 

•  Students  must  wait  for  resources  to  become  available  to  progress  to  the 
next  task.  Tabletop  or  lecture  based  workarounds  used  in  real  life  are  not 
used  in  the  model. 

•  The  model  duplicates  a  certain  percentage  of  each  class  to  simulate 
additional  training  requirements.  These  additional  students  represent 
constraints  on  resources  due  to  non-progression,  ineffective  sorties, 
unaccomplished  tasks,  etc.  The  model  does  not  count  these  duplicates 
when  calculating  delayed  students  or  graduation  statistics. 

•  A  student  is  limited  to  12  hours  a  day  to  complete  aircraft  training  and  8 
hours  for  GBTS  training. 
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•  Delays  due  to  weather  and  sickness  or  emergencies  will  not  be  simulated. 
It  is  assumed  that  the  syllabi  have  sufficient  flex  days  built  in  to 
compensate  and  weekends  would  be  used  if  necessary  to  maintain 
scheduled  graduation  dates. 

•  The  number  of  available  computers,  instructors,  classrooms  and  airfield 
resources  are  not  constraining  factors.  However,  classroom  instruction 
must  be  completed  within  the  12-hour  flying  day. 

•  All  students  will  complete  the  entire  training  program  without  dropping 
out  or  recycling  back  to  a  later  class. 

•  Current  syllabi  do  not  integrate  fifth  generation  requirements  or 
capabilities  throughout  the  course.  This  model  is  developed  in 
anticipation  of  updated  syllabi  and  will  serve  as  a  framework  for 
adjustments. 

•  The  model  structure  must  remain  simple,  easy  to  understand  and  edit,  and 
only  focus  on  a  small  number  of  decision  variables. 

With  these  assumptions  and  limitations,  the  methodology  still  captures  the  main 
effects  that  affect  aircraft  and  GBTS  resource  availability  and  effectiveness  during  pilot 
training. 

Course  Flow 

The  simulation  uses  syllabi  for  two  courses.  Introduction  to  Fighter  Fundamentals 
(IFF)  and  T-38C  Specialized  Undergraduate  Pilot  Training  (SUPT),  as  the  framework  for 
course  completion.  The  simulation  creates  students  as  a  class  and  pushes  each  class  of 
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students  through  the  training  together.  Within  each  syllabus  are  several  track  options  that 
a  student  can  take.  The  model  assigns  an  identifier  to  each  student  for  the  track  of 
instruction  the  student  will  follow  as  well  as  an  identifier  for  when  the  student  started 
training.  After  the  model  assigns  tracks,  the  students  flow  through  each  week  of 
instruction  until  the  course  is  complete.  After  completion  of  instruction  for  a  syllabus, 
the  model  records  how  many  students  have  graduated.  If  there  are  not  sufficient 
resources  to  graduate  each  student  on  time,  the  model  records  how  many  students 
graduated  one,  two,  three,  four  or  more  weeks  late.  In  addition  to  the  daily  training 
processes,  the  model  includes  processes  to  reduce  the  number  of  available  resources 
resulting  from  maintenance  requirements. 


A  Week’s  Schedule 

The  depiction  in  Figure  4  is  a  partial  representation  of  a  training  week.  Figure  4 
features  two  training  days  with  a  variety  of  aircraft  tasks,  a  GBTS  task,  an  academic  task, 
as  well  as  logic  used  to  manage  the  movement  of  students  along  syllabus  tracks. 


End  the  week 


Reset 

student 

count 


Figure  4.  A  Simulation  Week 
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For  each  training  week,  the  model  begins  by  counting  the  number  of  students  that 
will  process  through  that  week’s  activities  and  the  number  of  students  in  each  track.  The 
model  holds  students  until  the  model  timer  reaches  the  beginning  of  the  week.  After  the 
model’s  clock  reaches  the  start  of  the  first  day  of  the  week,  students  flow  through  their 
respective  tracks  and  complete  the  necessary  academic,  aircraft,  or  GBTS  training.  Once 
the  students  finish  the  first  day’s  tasks,  the  students  again  hold  until  the  clock  marks  the 
beginning  of  the  next  day.  If  the  new  day  has  already  begun  and  the  student  is  still 
completing  tasks  from  previous  days  due  to  a  lack  of  available  aircraft  or  GBTS,  the 
model  will  immediately  flow  the  student  to  the  next  day.  If  a  student  does  not  process 
through  each  day  before  the  weekend  arrives,  the  model  holds  that  student  in  the  current 
week  while  the  rest  of  the  class  flows  to  the  next  week.  In  order  to  allow  the  last  training 
day  of  each  week  to  have  flexibility  like  the  previous  days,  students  may  flow 
immediately  from  the  fourth  day  to  the  fifth  without  delay.  The  flexibility  between 
modeled  days  simulates  the  flexibility  instructors  have  to  adjust  scheduled  activities 
during  the  week  to  meet  syllabus  priorities  and  avoid  bottlenecks  caused  by  rigid  syllabus 
schedules.  After  the  end  of  the  week,  the  model  resets  all  the  variables  used  for  counting 
students  in  preparation  for  the  next  class.  After  the  model  completes  the  week,  it  pushes 
the  students  on  to  the  next  week’s  activities. 

Scheduled  Maintenance 

Aircraft  scheduled  maintenance  is  a  periodic  process  used  to  proactively  avoid 
problems  and  ensure  the  aircraft  is  in  working  order.  Specific  aircraft  components 
require  various  maintenance  schedules  since  not  all  components  are  as  robust  as  others 
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are.  Intervals  between  scheduled  maintenance  actions  can  be  based  on  a  number  of 
factors  such  as  number  of  uses,  hours  of  use,  fly  hours  flown,  days  since  last  inspection 
and  so  forth.  For  the  purpose  of  this  study,  the  number  of  flying  hours  since  the  last 
scheduled  maintenance  event  defines  the  interval  time  between  scheduled  maintenance. 
The  user  inputs  a  value  into  the  variable  AcftTimeBetweenScheduIeMx  to  set  how  many 
flying  hours  the  model  counts  before  schedule  maintenance  on  an  aircraft  occurs. 

The  user  defines  the  amount  of  time  an  aircraft  may  spend  in  scheduled 
maintenance  based  on  a  triangular  distribution.  The  user  inputs  the  time-to-fix  into  the 
variables  AcftScheduleMxTimeLow,  AcftScheduIeMxTimeAvg,  and 
AcftScheduleMxTimeHigh.  These  times  represent  the  total  time  an  aircraft  is  not 
available  to  fly  because  of  actions  associated  with  scheduled  maintenance.  This 
stipulation  assumes  the  user  factors  in  maintenance  delays  and  logistics  delays  into  the 
fix -time.  A  triangular  distribution  is  appropriate  for  this  model  because  prospective 
aircraft  may  not  have  sufficient  data  to  establish  a  known  distribution  (Banks  et  ah, 

2010: 183).  The  triangular  distribution  is  also  beneficial  because  it  can  bound  known 
minimum  and  maximum  durations  while  still  allowing  the  flexibility  to  include  a  tail  for 
unexpected  delays.  For  the  purpose  of  scenarios  analysis,  this  study  sets  the  low, 
average,  and  high  values  of  scheduled  maintenance  equal  to  each  other  since  appropriate 
distribution  data  was  not  available.  Unscheduled  maintenance  actions  will  be  explained 
later  in  the  thesis. 
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GBTS  Maintenance 


Maintenance  actions  for  the  ground  based  training  systems  are  not  based  on  a 
scheduled  or  unscheduled  maintenance  action  like  aircraft  maintenance.  This  study 
assumes  that  all  of  the  GBTSs  are  stationary  flight  simulators  and  system  failures  will 
resemble  those  of  a  network  of  computers.  Computer  failures  are  represented  as  rates 
that  measure  the  number  of  failures  in  a  given  sample  size  over  a  specified  period  of  time 
(Schroeder  and  Gibson,  2007:1).  As  such,  it  is  appropriate  to  represent  GBTS  failure 
rates  and  variable  fix  times  with  the  overall  mission  capable  rate  of  the  system  rather  than 
on  some  level-of-use  metric. 

The  decision  maker  provides  the  mission  capable  rate  of  the  GBTS  as  the  percent 
of  time  a  system  is  available  for  use,  represented  by  the  variable  SimAo.  The  simulation 
randomly  selects  each  GBTS  each  day  and  holds  the  asset  for  a  portion  of  the  day  not 
included  in  SimAo.  As  an  example,  to  represent  maintenance  requirements  the  model 
would  randomly  pull  a  GBTS  and  hold  it  for  20  percent  of  the  day  if  the  GBTS  mission 
capable  rate  was  80  percent. 

Academic  Instruction 

Pilot  training  divides  academic  instruction  between  instructor  based  class 
instructions,  ground  training  and  computer  based  instruction.  The  focus  of  the  research  is 
on  GBTS  and  aircraft  requirements,  so  resources  associated  with  academic  training  are 
not  constrained.  Although  academic  resources  are  not  a  factor,  the  time  required  to 
complete  academic  training  constrains  the  student  with  fewer  available  hours  to  complete 
GBTS  and  aircraft  training.  As  a  result,  the  model  treats  academics  as  a  simple  delay  in 
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order  to  model  the  constraint  on  available  time  to  complete  other  activities.  The  model 
holds  each  student  for  the  duration  of  the  class  time  before  moving  the  student  on  to  the 
next  task. 

Aircraft  Training 

The  simulation  models  aircraft  training  events  with  a  submodel  consisting  of  three 
major  functions,  batching,  training  time  and  aircraft  maintenance.  Figure  5  depicts  the 
three  functions  with  how  they  work  detailed  below.  Figure  5  is  the  most  robust  version 
of  aircraft  simulation  logic  and  can  handle  up  to  four  student  pilots  with  any  number  of 
aircraft.  For  training  tasks  that  require  less  than  four  students,  the  model  uses  simplified 
versions  of  Figure  5  that  require  less  coding  for  the  batching  and  maintenance  portions  of 
the  submodel. 


Training 


Aircraft  Training-Batching 

The  purpose  of  batching  is  to  simulate  aircraft  training  events  that  require 
multiple  students  to  train  together.  The  batching  process  compares  the  number  of 
students  that  will  pass  through  the  event  that  day  with  the  number  of  students  the  event 
requires  to  train  together.  The  model  will  count  out  the  students  as  they  arrive  and 
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combine  them  into  a  single  batch,  sized  by  the  user’s  input.  If  there  are  not  enough 
students  to  fill  an  entire  batch,  the  remaining  students  form  a  smaller  partial  batch. 

Aircraft  Training-Training  Time 

After  a  group  of  students  is  batched,  the  batched  students  complete  the  training 
together.  If  the  required  aircraft  are  not  available,  the  group  of  students  will  wait  until 
enough  resources  become  available  to  complete  the  training.  Normally  the  students 
would  conduct  a  mission  brief,  preflight  check,  fly  the  training  mission,  debrief  and  then 
continue  to  the  next  training  event  for  that  day.  The  top  half  of  Figure  6  reflects  this 
order. 

In  order  to  capture  the  unique  processes  of  aircraft  and  students,  the  model 
sequences  events  to  follow  the  path  in  the  bottom  half  of  Figure  6.  The  model  seizes 
aircraft  and  instructors  at  the  same  time  and  simulates  flying  the  training  mission.  From 
this  point,  the  model  splits  the  processes  into  student  and  aircraft  specific  tasks.  Students 
experience  time  delays  to  simulate  mission  briefs  as  well  as  debriefs.  The  mission  brief 
delay  begins  at  a  specified  amount  of  time  prior  to  flight  takeoff  and  encompasses  a  delay 
for  preflight  checks.  At  the  end  of  the  training  time,  the  model  delays  the  student  for  the 
required  time  a  student  must  wait  post-flying  before  starting  another  training  event. 
Aircraft  experience  time  delays  to  simulate  preflight  checks  and  maintenance  actions. 
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Figure  6.  Aircraft  Training  Process 


By  using  this  modified  sequence,  the  model  can  link  student  and  resource 
requirements  in  a  less  complex  fonnat  while  maintaining  fidelity.  This  format  maintains 
the  time  intervals  between  successive  events  for  both  students  and  aircraft.  This  format 
also  synchronizes  the  resources  in  a  way  that  is  flexible  enough  for  the  user  to  add 
constraints  on  instructor  pilot  availability  in  future  research.  Specifically,  this  format 
avoids  situations  where  students  seize  instructors  for  the  mission  brief  and  then  hold  them 
for  several  hours  while  waiting  for  aircraft  to  become  available. 

Aircraft  Training- Aircraft  Maintenance 

Following  a  training  mission,  maintenance  personnel  check  the  aircraft  to  ensure 
proper  functionality.  If  the  aircraft  lands  with  inoperable  mission-essential  systems,  it 
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becomes  “Code-3”  and  requires  unscheduled  maintenance  (AFP AM  63-128,  2009:157). 
Code-3  conditions  also  include  ground  aborts  (AFP AM  63-128,  2009: 157).  Once  the 
inoperable  system  is  fixed,  the  aircraft  is  prepared  for  the  next  training  mission. 

The  model  determines  which  aircraft  need  unscheduled  maintenance  based  on  a  break 
rate  percentage.  The  variable  AcftBreakRate  is  calculated  as  the  percentage  of  Code  3 
breaks  per  training  sorties  flown.  After  aircraft  complete  the  training  mission,  the  model 
probabilistically  selects  which  ones  will  require  maintenance.  The  duration  of  time  an 
aircraft  spends  in  maintenance  depends  on  an  eight-hour  fix  rate  with  a  gamma 
distribution.  The  eight-hour  fix  rate  represents  the  percentage  of  aircraft  that  land  Code-3 
and  can  be  fixed  within  eight-hours.  The  fix  rate  is  an  appropriate  metric  because  it  is  a 
maintainability  measure  that  combines  logistics  and  administrative  delays  with  the  direct 
maintenance  time  (AFP AM  63-128,  2009:158).  This  rate,  and  an  estimated  shape 
parameter,  act  as  inputs  to  an  excel  solver  function  to  generate  the  scale  parameter  of  the 
gamma  distribution  as  well  as  the  mean  fix  time.  The  gamma  distribution  is  an  adequate 
application  for  simulating  machine  repair-time  variability  (Law,  2007:285).  The  model 
delays  the  aircraft  resource  for  the  duration  of  time  detennined  by  the  gamma 
distribution. 

A  series  of  logic  blocks  determine  when  the  aircraft  began  maintenance,  ended 
maintenance,  and  if  any  of  the  time  in  between  spanned  maintenance  off  hours  or 
weekends.  If  any  of  the  delay  time  coincides  with  weekends  or  maintenance  off  hours, 
the  model  adds  this  time  to  the  original  delay.  The  reason  the  model  utilizes  test  logic 
instead  of  resource  schedules  is  because  of  Arena®  functionality  errors.  Normally  a 
schedule  with  preempt  delays  would  sufficiently  add  down  time  to  the  required  fix  time 
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in  the  event  the  fix  time  spanned  a  weekend  or  maintenance  off  hours.  Verification 
testing  identified  a  glitch  in  this  functionality,  however,  and  forced  the  utilization  of 
cascading  logic  functions  as  a  workaround.  During  verification  tests,  the  preempt  feature 
only  operated  as  advertised  when  the  model  had  fewer  than  three  students  to  process. 
With  more  than  three  students  however,  maintenance  actions  continued  to  process  instead 
of  delaying  the  fix  until  maintenance  resources  again  became  available. 

After  the  appropriate  amount  of  time,  the  model  releases  the  aircraft  from 
maintenance  work,  or  bypasses  the  process  if  no  breaks  occur.  Following  corrective 
maintenance  actions,  the  aircraft  begins  preparation  for  the  next  mission.  The  mission 
preparation  delay  is  the  average  time  required  to  turn  the  jet.  This  average  time  is 
defined  by  the  user  set  variable  AcftTurntime. 

Weapon  Systems  Operators 

Most  aircraft  training  events  follow  the  same  structure  mentioned  above  with 
batching,  training  time,  and  maintenance  elements.  The  exceptions  to  this  fonnat  are 
aircraft  training  events  designed  for  weapon  systems  operator  (WSO)  students.  For 
dedicated  training  events,  the  WSO  student  has  his/her  own  jet.  In  these  cases,  the  event 
follows  the  format  outlined  above.  In  non-dedicated  training  events,  the  WSO  flies  in  the 
back  seat  of  the  aircraft  while  another  student  pilot  conducts  aircraft  training. 

The  training  syllabus  maps  WSO  students  on  their  own  track,  even  though  the 
WSO  student  is  dependent  on  the  other  tracks  to  complete  training  (AETC,  2013:  62). 

As  a  result,  the  syllabus  course  flow  may  not  be  possible  to  follow  if  the  other  tracks  do 
not  have  sufficient  student  pilots  on  a  given  track.  In  the  real  world,  the  WSO  track 
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schedule  synchronizes  with  the  other  tracks  in  a  given  class  so  that  flying  tasks  match  up 
on  the  same  days.  After  synchronization,  the  non-dedicated  WSO  training  does  not 
constrain  any  resources  beyond  what  the  student  pilots  are  already  using.  To  capture  this 
effect,  the  model  treats  non-dedicated  WSO  events  as  simple  delays  that  do  not  seize  any 
resources.  Instead,  the  WSO  student  holds  for  amount  of  time  to  cover  the  brief  and 
flight  check,  flying  training,  debrief,  and  minimum  student  wait  time.  With  this  process, 
the  model  still  tracks  the  number  of  resources  required  and  the  number  of  hours  left  in  the 
day  to  complete  other  training  events. 

GBTS  Training 

Training  with  ground  based  trainer  systems  follows  the  same  basic  flow  as 
training  with  an  aircraft.  The  student  has  a  prebrief  with  the  instructor,  flies  the  training 
mission,  debriefs  and  moves  on  to  the  next  task.  The  model  structure  to  simulate  GBTS 
training  is  only  slightly  different  from  the  structure  for  aircraft  training. 

The  batching  logic  in  the  GBTS  submodel  is  identical  to  the  batching  logic  in  the 
aircraft  submodel  explained  above.  Although  most  GBTS  events  are  only  single  student 
events,  the  model  is  scalable  to  accommodate  multiple  students  training  together  on 
linked  systems.  Unlike  the  maintenance  portion  of  the  aircraft  submodel,  the  GBTS 
maintenance  portion  does  not  include  a  process  for  unscheduled  maintenance.  The 
GBTS  Maintenance  section  of  this  thesis  explains  how  the  model  simulates  scheduled 
and  unscheduled  maintenance  requirements.  The  maintenance  section  of  the  submodel 
does  however  include  the  necessary  delay  to  set  up  the  GBTS  for  the  next  training 
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scenario.  The  training  time  portion  of  the  submodel  is  also  similar  to  the  aircraft 
submodel’s  training  time  except  there  is  no  extra  delay  included  for  preflight  checks. 

Additional  Training  Requirements 

Pilot  training  involves  many  difficult  tasks  that  very  few  people  master  the  first 
time.  Whether  the  student  is  not  learning  the  material  quickly  enough,  or  the  aircraft  is 
not  functioning  properly,  additional  sorties  are  often  required  to  complete  training.  To 
capture  the  strain  on  resources  that  additional  sorties  create,  the  model  creates  a  number 
of  shadow  students  to  process  with  each  class.  The  user  detennines  what  percent  of 
aircraft  or  GBTS  sorties  require  repetition.  If  a  class  of  ten  students  can  pass  a  scheduled 
lesson  with  one  sortie  each,  then  no  shadow  students  are  required.  If  the  class  requires  an 
additional  three  sorties  for  training  or  to  make  up  for  maintenance  problems,  however, 
the  model  would  need  three  shadow  students  to  perfonn  the  additional  sorties.  For  the 
purposes  of  this  study,  the  model  uses  a  ten  percent  additional  sortie  requirement  based 
on  planning  factors  stated  in  the  syllabus  (AETC,  2013:3).  After  the  model  generates 
entities  to  represent  each  student  it  assigns  the  students  to  specific  tracks  of  instruction. 
After  students  are  in  their  tracks,  the  model  replicates  ten  percent  of  them  and  assigns  an 
attribute  to  the  replicated  entities  to  identify  them  as  shadow  students.  These  shadow 
students  create  the  additional  requirements  for  aircraft  and  GBTS  but  the  model  does  not 
include  them  when  calculating  statistics  for  student  throughput. 


25 


IV.  Analysis  and  Results 


Verification  and  Validation 

Prior  to  conducting  tests  and  analysis,  model  verification  ensures  that  the  model 
functions  as  the  user  would  expect.  This  study  utilized  several  verification  techniques  to 
build  confidence  in  the  model’s  ability  to  guide  insight.  In  order  to  avoid  the  occurrence 
of  typographical  errors  in  the  logic  throughout  the  model,  variable  names  pulled  from 
dropdown  menus  were  used  extensively.  When  replicating  submodels,  the  previous 
submodel  provided  the  framework  for  the  following  submodel.  This  copy-and-paste 
process  forced  the  model  builder  to  inspect  each  process  for  correctness  prior  to  updating 
the  information  for  the  following  submodel.  In  addition  to  inspection  of  each  module’s 
logic,  visual  analysis  of  process  flows  ensured  that  entities  flowed  through  the  model 
appropriately.  Through  use  of  the  animation  function  in  Arena®,  each  type  of  submodel 
was  inspected  prior  to  inclusion  with  the  rest  of  the  model.  The  use  of  test  scenarios 
combined  with  the  statistical  data  produced  by  Arena®  output  reports,  further  validated 
that  model  logic  functioned  as  intended. 

Along  with  verification,  validation  ensures  the  model  will  provide  usable 
infonnation  to  decision  makers.  While  verification  ensures  the  model  is  running 
properly,  validation  ensures  the  model  sufficiently  represents  the  real  system  (Carson, 
2002:52).  Input  from  subject  matter  experts  guided  the  development  of  the  model’s 
weekly  training  processes  as  well  as  the  training  submodel  structures.  Discussions  with 
maintenance  professionals  and  pilot  instructors  ensured  that  the  model  included 
applicable  elements  of  the  real  world.  In  addition  to  input  from  subject  matter  experts, 
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pilot  training  syllabi  and  data  from  the  Air  Force  Training  Information  Management 
System  provided  input  for  process  times,  resource  schedules,  and  activity  delays  used 
throughout  the  model.  Further  validation  of  the  model  occurred  through  a  comparison  of 
model  outputs  generated  by  the  Arena®  process  analyzer  tool.  Table  1  below  compares 
real  world  data  for  annual  sortie  generation  and  flying  hours  with  model  outputs.  The 
statistics  for  current  data  reflect  the  minimum  fly  hours  and  sorties  generated  for  an 
average  of  100  students  in  a  year.  When  no  extra  shadow  students  are  included,  the 
model  outputs  the  second  line  of  statistics  in  Table  1.  This  data  reflects  the  minimum 
flying  hours  and  sorties  required  when  no  less  than  100  students  are  expected  to  graduate 
and  absolutely  no  additional  sorties  are  required.  Line  three  shows  the  surge  capacity 
statistics  for  worst-case  scenario  planning.  This  study  evaluates  the  worst-case  scenario 
with  the  assumption  that  the  Air  Force  will  want  to  maintain  current  surge  capabilities. 


Table  1.  Annual  Fly  Hours  and  Sorties  (Factsheet,  2012) 


# 

Annual  Fly  Hours 

# 

Annual  Sorties 

Current 

>13,000 

>11,500 

Model  w/o 

10%  increase 

13,391-13,399* 

11,284-11,276* 

Model  w/ 

10%  increase 

14,583-14806* 

12,280-12,467* 

*95%  confidence  intervals 

Replication  Parameters 

The  scenarios  in  this  study  analyze  both  stochastic  and  non-stochastic  elements  of 
the  pilot  training  process.  Aircraft  break  rate,  8-hr  fix  rate,  non-scheduled  aircraft 
maintenance  time,  additional  training  requirements,  student  training  tracks  and  GBTS 
availability  all  add  randomness  to  the  model.  Other  elements  such  as  scheduled 
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maintenance  time,  aircraft  and  GBTS  turn  times,  training  times,  briefing  times,  and  class 
sizes  are  assumed  constant  in  this  model.  In  order  to  obtain  meaningful  information  from 
both  stochastic  and  nonstochastic  elements,  the  scenarios  in  this  study  follow  specific 
replication  parameters. 

The  model  uses  a  one-year  wann  up  period  and  a  year  long  run  time  to  produce 
scenario  results.  Six  months  are  required  to  populate  the  SUPT  training  curriculum  with 
students  in  training  stages  from  start  to  finish.  Until  the  entire  course  initializes,  the 
resources  are  not  fully  constrained.  Another  six  months  are  required  to  fully  constrain 
the  resources  as  they  currently  are  in  the  real  world.  After  a  year  of  initialization,  entities 
have  both  filled  the  entire  model  and  produced  the  appropriate  constraints  on  resources. 

A  year’s  worth  of  students  then  process  through  the  model  to  measure  the  effects  of 
course  requirements  and  resource  availability  on  student  throughput. 

The  model  runs  each  scenario  fifty  times  to  generate  the  appropriate  data  in 
analysis.  By  increasing  the  number  of  replications,  confidence  intervals  decrease  to 
allow  better  representation  of  data.  Confidence  interval  half-length  analysis,  detennines 
the  appropriate  number  of  replications  required  to  reach  a  desired  confidence  interval 
span  (Banks  et  ah,  2010:43 1).  Some  parameters  in  this  study  can  be  determined  with  10 
or  20  replications,  however  MTBM  requires  fifty  replications  to  gain  accuracy  within  ±.  1 
maintenance  actions/fly  hours  at  the  0.05  significance  level.  Fifty  replications  also 
produce  confidence  intervals  within  ±139  and  ±1 17,  at  the  0.05  significance  level,  for 
number  of  annual  aircraft  flying  hours  and  number  of  annual  aircraft  sorties  respectively. 
With  these  replications,  the  average  number  of  aircraft  sorties  per  scheduled  aircraft  per 
day  is  accurate  to  within  ±0.03  sorties  with  a  95%  confidence  interval. 
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Baseline  Single  Base  SUPT  Scenario 

Analysis  began  with  a  baseline  scenario  that  used  the  resources  available  at  a 
representative  base.  Laughlin  AFB  and  its  associated  SUPT  curriculum  served  as  the 
baseline  scenario.  A  total  of  65  aircraft  and  7  GBTS  with  a  student  throughput  of  103 
students  per  year  characterize  the  scenario  environment.  In  order  to  maintain  surge 
capacity,  evaluations  focused  on  supporting  120  students  rather  than  103.  The  model 
utilizes  the  threshold  values  stated  in  the  T-X  presolicitation  documents  to  initialize 
variables  for  aircraft  and  GBTS  perfonnance  factors.  The  minimal  performance  factors 
and  system  attributes  used  in  the  model  are  listed  on  FedBizOpps.gov  and  displayed  in 
Table  2  below  (Christian,  2013).  The  baseline  includes  an  additional  ten  percent  shadow 
student  increase  to  each  class  to  simulate  refly  requirements  that  can  be  expected  to  occur 
(AETC,  2013:3).  These  threshold  attributes  represent  the  current  capability  of  the  T-38 
family  of  systems  and  the  minimum  level  of  performance  that  the  Air  Force  requires  the 
T-X  FoS  to  meet  (Christian,  2013). 


Table  2.  Draft  Key  Perfonnance  Factors  and  Key  System  Attributes  (Christian,  2013) 


Attribute 

Description 

Threshold 

GBTS-Ao 

Percentage  of  time  GBTS  are  operationally 
capable  of  perfonning  mission 

No  less  than  80% 

Aircraft-Ao 

Percentage  of  time  aircraft  are  operationally 
capable  of  perfonning  mission 

No  less  than  64.7% 

Aircraft 

Turn- Around 
Time 

Mean  time  it  takes  to  recover  an  aircraft  and 
complete  any  preparation  needed  to  ready  the 
aircraft  for  the  next  mission 

No  greater  than  0.75 
hours 

Aircraft 

Break  Rate 

Percentage  of  aircraft  that  land  Code-3 

No  greater  than  8% 

Aircraft  Fix 
Rate 

Percentage  of  Aircraft  that  land  Code-3  and 
can  be  fixed  within  8  hours 

No  less  than  75% 

Aircraft 

MTBM 

Mean  time  between  corrective  and  preventative 
maintenance  actions 

No  less  than  14.71 
flying  hours 

29 


In  order  to  evaluate  whether  or  not  the  scenario  can  support  the  required  annual 
student  graduation  rate,  an  analysis  of  model  output  data  is  required.  The  Arena® 
process  analyzer  tool  allows  the  user  to  manipulate  a  vast  array  of  model  inputs  and 
generates  many  output  statistics.  With  the  simulation-produced  statistics,  the  study  uses 
Excel  spreadsheets  to  produce  additional  statistics  for  MTBM,  #  sorties  per  resource,  and 
aircraft  availability.  Figure  7  below  highlights  some  of  the  statistics  of  interest  for  this 
study  as  well  as  the  results  of  the  analysis  process  with  baseline  scenario  inputs. 

Although  the  number  of  aircraft  and  GBTS  function  as  inputs  for  the  simulation  model, 
the  number  of  resources  required  to  support  120  student  graduates  annually  are  an  output 
generated  by  the  analysis  process. 


141  flyhrs 

1440  hrs 

MTBSM 

SMxTime 

8% 

Break  Rate 

75% 

Fix  Rate 

0.75hrs 

Turn  Time 

80% 

GBTS-Ao 

#Acft 
#  GBTS 
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Acft-Ao 

64.67% 

±  0.27 

MTBM 

13.35  flyhrs 
±  0.09 

#  Sorties/ 

Scheduled  Acft/Day 

2.01 

±0.02 

#  Sorties/ 

Ao  GBTS/Day 

2.84 

±0.02 

Figure  7.  Baseline  Scenario  w/120  Graduates 


The  data  in  Figure  7  highlights  that  the  model  was  able  to  graduate  120  students 
on  time.  When  determining  the  appropriate  number  of  aircraft  and  GBTS  required  to 
support  pilot  training,  this  study  will  only  accept  scenarios  where  all  of  the  students  are 
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able  to  graduate  on  time.  Maintaining  one  hundred  percent  on  time  graduation  not  only 
holds  true  to  the  baseline  capability,  it  also  avoids  the  issue  of  deciding  how  many 
delayed  students  are  too  many. 

Aircraft  availability  is  a  function  of  aircraft  maintenance  times  and  frequency  of 
scheduled  or  unscheduled  maintenance  actions  as  a  result  fleet  flying  hours.  It  is 
important  to  note  that  aircraft  availability  is  an  output  statistic  dependent  on  flying  hours 
and  maintenance  actions.  Scenarios  with  different  number  of  aircraft  or  different  number 
of  flying  hours  will  have  different  availability  rates  even  though  the  maintenance 
requirements  are  the  same.  In  early  stages  of  this  study,  the  model  employed  aircraft 
availability  rates  as  an  input,  much  like  the  GBTS.  This  fonnat  skewed  the  data  and  gave 
the  impression  that  far  fewer  aircraft  than  were  actually  necessary  could  support  pilot 
training  requirements.  When  developing  scenarios,  this  study  uses  the  threshold 
attributes  to  establish  the  aircraft  availability  at  the  most  stringent  point  before  relaxing 
constraints. 

The  model  outputs  utilization  rates  for  both  aircraft  and  GBTS.  These  rates  are  a 
combination  of  training  use  as  well  as  maintenance  use.  These  statistics  are  important 
tools  that  help  decision  makers  understand  not  only  how  many  resources  are  required  to 
complete  the  mission,  but  also  how  busy  those  resources  are  on  a  daily  basis.  Equation  1 
below  outlines  how  data  taken  from  the  process  analyzer  was  manipulated  to  obtain 
number  of  daily  sorties  each  aircraft  on  the  flying  schedule  would  have  to  fly  in  order  to 
meet  demand  requirements.  This  study  assumes  that  of  the  unit  assigned  aircraft  that  are 
mission  capable,  75  percent  will  be  available  for  use  on  a  given  day.  In  accordance  with 
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maintenance  practices,  the  other  25  percent  are  held  in  reserve  to  ensure  enough  mission 
capable  aircraft  are  ready  the  following  day. 


(JJa  - 1  +  Ao)  -A- 24 -7 
~  (T  +  F  +  Q-5-A-Ao-Sc 


(1) 


Where: 

S  =  Average  number  of  sorties  per  day  per  scheduled  aircraft 
Ua  =  Total  aircraft  utilization 

Ao  =  Percentage  of  time  aircraft  are  operationally  capable  of  performing  mission 
A  =  Number  of  aircraft  assigned 
T  =  Aircraft  turn  time 
F  =  Average  sortie  duration 
C  =  Flight  check  time 

Sc  =  Percent  of  mission  capable  aircraft  placed  on  the  flying  schedule 


With  data  generated  by  the  baseline  scenario  of  the  model,  the  above  equation 
shows  that  3 1  scheduled  aircraft  would  fly  an  average  of  2.01  sorties  each  day  to  support 
the  surge  student  throughput. 

The  metric  for  mean  time  between  maintenance  (MTBM)  proved  to  be 
problematic  for  this  study.  At  the  threshold  performance  levels,  the  baseline  scenario 
was  not  able  to  meet  the  minimum  requirement  of  no  fewer  than  14.71  flying  hours 
between  maintenance  actions.  Even  when  excluding  scheduled  maintenance  actions,  the 
model  could  barely  meet  MTBM  threshold  levels  when  the  aircraft  break  rate  was  at 
threshold  levels.  In  order  to  satisfy  the  threshold  requirement  at  the  objective  level  break 
rate,  the  average  time  between  scheduled  maintenance  would  have  to  exceed  230  flying 
hours.  At  this  interval,  each  scheduled  maintenance  action  would  average  in  excess  of  90 
days.  Most  scheduled  maintenance  actions  on  the  T-38  are  minor  periodic  inspections 
that  range  from  a  week  to  a  month  in  duration  (Maysonet,  2013).  More  extensive  and 
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less  frequent  inspections  last  for  45  days  (Maysonet,  2013).  The  fact  that  the  model 
would  require  an  aircraft  to  receive  scheduled  maintenance  less  than  twice  a  year  for  over 
90  days  suggests  that  the  MTBM  metric  or  the  aircraft  break  rate  or  the  aircraft  fix  rate 
metric  may  not  reflect  real  world  values. 

Future  State  Scenarios 

With  the  background  infonnation  gained  from  the  baseline  test,  decision  makers 
are  able  to  test  possible  future  scenarios.  A  comparison  of  both  results  shows  the  impact 
of  future  changes  on  the  current  state  of  operations.  Currently  the  Air  Force  is 
considering  purchasing  350  T-X  aircraft  to  replace  the  aging  T-38  fleet.  Although  the 
new  aircraft  will  be  easier  to  maintain,  the  new  fleet  would  still  constitute  more  than  an 
eighteen  percent  reduction  in  available  aircraft.  The  future  state  scenarios  in  this  study 
begin  with  53  aircraft  instead  of  the  current  65  to  represent  the  proportional  reduction  in 
the  fleet.  Tables  three  through  five  below  show  the  results  of  possible  future  state 
scenarios.  Each  table  shows  the  effects  of  shifting  FoS  attributes  from  threshold  levels  to 
objective  levels  while  maintaining  the  same  scheduled  maintenance  plan  as  well  as  the 
same  number  of  aircraft  and  GBTS.  Between  the  three  tables,  the  scheduled  maintenance 
plan  adjusts  the  aircraft  availability. 
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Table  3.  Attribute  Trade  Space  with  64.7%  Mission  Capable  Minimum 


53  Acft,  7  GBTS,  141  MTBSM,  1163  SMx  Time 

Input 

Results 

Break 

Fix 

Turn 

GBTS 

% 

MTBM 

# 

# 

Rate 

Rate 

Time 

-Ao 

Acft- 

Sorties/ 

Sorties/ 

Ao 

Scheduled 

Ao 

Acft/ 

GBTS/ 

Day 

Day 

T 

T 

T 

T 

64.68 

13.42 

2.46 

2.83 

T 

T 

T 

0 

64.52 

13.43 

2.47 

2.53 

T 

T 

0 

T 

64.72 

13.39 

2.46 

2.83 

T 

T 

0 

0 

64.87 

13.45 

2.45 

2.51 

T 

0 

T 

T 

64.78 

W 

2.47 

2.84 

T 

0 

0 

T 

64.93 

■W 

2.46 

2.83 

T 

0 

T 

0 

65.03 

■HEMi 

2.45 

2.51 

T 

0 

0 

0 

64.89 

w®m 

2.47 

2.52 

0 

T 

T 

T 

64.96 

■BEK 

2.45 

2.84 

0 

T 

T 

0 

64.77 

■HEM 

2.47 

2.54 

0 

T 

0 

T 

64.83 

— 

2.48 

2.85 

0 

T 

0 

0 

64.87 

2.47 

2.53 

0 

0 

T 

T 

65.02 

2.46 

2.83 

0 

0 

T 

0 

64.99 

2.47 

2.53 

0 

0 

0 

T 

65.1 

2.46 

2.83 

0 

0 

0 

0 

65.04 

2.47 

2.52 

95%  Confidence  Intervals 

±0.3 

±0.11 

±0.03 

±0.02 

T=  Threshold,  0=  Objective, 

MTBSM=Mean  Time  Between  Scheduled  Maintenance 

SMx  Time=  Mean  Scheduled  Mx  Time 

The  data  in  Table  3  shows  that  even  with  an  eighteen  percent  reduction  in  aircraft, 
the  base  fleet  would  still  be  able  to  support  pilot  training  requirements  at  a  surge  capacity. 
The  very  small  variability  in  response  factors  also  indicates  that  fluctuations  between 
threshold  and  objective  levels  of  performance  factors  have  no  significant  impact  on  how 
many  resources  are  required  to  accomplish  training  requirements.  The  largest  variability 
resides  in  the  MTBM  statistics,  which  still  show  that  threshold  levels  of  break  rates  do 
not  meet  threshold  requirements  for  MTBM.  MTBM  is  most  responsive  to  shifts  from 
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objective  to  threshold  levels  of  break  rate.  The  aircraft  availability  metric  however,  is 
relatively  unchanged  by  variations  in  break  rates.  These  relationships  show  that  although 
aircraft  breaks  make  up  the  majority  down  time  events,  the  breaks  are  fixed  fast  enough 
to  have  minimal  impact  on  aircraft  availability  while  the  scheduled  maintenance  actions 
have  significant  impact  on  aircraft  availability. 

Most  notable  from  the  results  in  Table  3,  is  the  increase  in  the  average  number  of 
required  daily  aircraft  sorties  per  scheduled  aircraft.  With  a  reduced  fleet  size,  the 
available  aircraft  must  assume  additional  demand  to  complete  the  same  load  of  work.  At 
the  threshold  64.7%  availability  level,  a  fleet  of  53  aircraft  will  only  have  25  aircraft  on 
the  flying  schedule  on  a  given  day.  The  result  is  an  average  22.39%  increase  in  number 
of  daily  sorties  each  scheduled  aircraft  will  have  to  complete. 
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Table  4.  Attribute  Trade  Space  with  70%  Mission  Capable  Minimum 


53  Acft,  7  GBTS,  141  MTBSM,  980  SMx  Time 

Input 

Results 

Break 

Fix 

Turn 

GBTS 

% 

MTBM 

# 

# 

Rate 

Rate 

Time 

-Ao 

Acft- 

Sorties/ 

Sorties/ 

Ao 

Scheduled 

Ao 

Acft/ 

GBTS/ 

Day 

Day 

T 

T 

T 

T 

69.93 

13.36 

2.27 

2.83 

T 

T 

T 

0 

69.95 

13.36 

2.27 

2.52 

T 

T 

0 

T 

70 

13.39 

2.27 

2.83 

T 

T 

0 

0 

70.02 

13.42 

2.27 

2.52 

T 

0 

T 

T 

70.14 

2.27 

2.84 

T 

0 

0 

T 

70.03 

2.29 

2.84 

T 

0 

T 

0 

69.94 

2.29 

2.54 

T 

0 

0 

0 

70.13 

2.28 

2.52 

0 

T 

T 

T 

70.18 

2.27 

2.84 

0 

T 

T 

0 

70.17 

2.28 

2.53 

0 

T 

0 

T 

70.14 

2.28 

2.84 

0 

T 

0 

0 

70.36 

2.26 

2.51 

0 

0 

T 

T 

70.24 

2.27 

2.83 

0 

0 

T 

0 

70.47 

2.25 

2.51 

0 

0 

0 

T 

70.54 

2.25 

2.82 

0 

0 

0 

0 

70.43 

2.26 

2.52 

95%  Confidence  Intervals 

±0.2 

±0.12 

±0.02 

±0.02 

T=  Threshold,  0=  Objective, 

MTBSM=Mean  Time  Between  Scheduled  Maintenance 

SMx  Time=  Mean  Scheduled  Mx  Time 
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Table  5.  Attribute  Trade  Space  with  80%  Mission  Capable  Minimum 


53  Acft,  7  GBTS,  141  MTBSM,  630  SMx  Time 

Input 

Results 

Break 

Fix 

Turn 

GBTS 

% 

MTBM 

# 

# 

Rate 

Rate 

Time 

-Ao 

Acft- 

Sorties/ 

Sorties/ 

Ao 

Scheduled 

Ao 

Acft/ 

GBTS/ 

Day 

Day 

T 

T 

T 

T 

80 

13.44 

2.84 

T 

T 

T 

0 

80.07 

13.41 

2.00 

2.51 

T 

T 

0 

T 

79.96 

13.41 

2.00 

2.84 

T 

T 

0 

0 

80.01 

13.41 

2.00 

2.52 

T 

0 

T 

T 

80.1 

13.38 

2.00 

2.85 

T 

0 

0 

T 

80.18 

13.39 

2.00 

2.83 

T 

0 

T 

0 

80.06 

2.00 

2.53 

T 

0 

0 

0 

80.19 

2.00 

2.52 

0 

T 

T 

T 

80.21 

2.00 

2.84 

0 

T 

T 

0 

80.25 

2.00 

2.52 

0 

T 

0 

T 

80.2 

2.00 

2.84 

0 

T 

0 

0 

80.22 

2.00 

2.53 

0 

0 

T 

T 

80.27 

2.00 

2.85 

0 

0 

T 

0 

80.45 

2.51 

0 

0 

0 

T 

80.37 

2.83 

0 

0 

0 

0 

80.3 

2.54 

95%  Confidence  Intervals 

±0.15 

±0.11 

±0.02 

±0.02 

T=  Threshold,  0=  Objective, 

MTBSM=Mean  Time  Between  Scheduled  Maintenance 

SMx  Time=  Mean  Scheduled  Mx  Time 

Table  4  and  Table  5  above  support  the  same  conclusions  as  Table  3.  The 
availability  of  aircraft  and  the  daily  strain  on  scheduled  resources  does  not  significantly 
vary  within  the  trade  space  of  the  measured  performance  levels  when  the  scheduled 
maintenance  plan  is  constant.  The  results  again  highlight  that,  at  threshold  levels  of 
aircraft  break  rate,  the  threshold  level  of  MTBM  is  not  realistically  obtainable.  At 
objective  levels  of  aircraft  break  rate,  the  objective  level  of  MTBM  is  impossible  to 
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reach.  The  scheduled  maintenance  plan  utilized  in  these  scenarios  produce  an  average 
MTBM  of  15.1 1  flying  hours.  If  no  scheduled  maintenance  actions  occurred,  the  average 
would  still  only  be  15.71  hours.  With  a  fleet  of  53  aircraft,  the  availability  of  resources 
greatly  depends  on  the  scheduled  maintenance  plan. 

When  the  scheduled  maintenance  plan  adjusts  to  increase  the  fleet  mission 
capable  rate,  more  aircraft  are  generated  for  scheduled  sorties.  With  a  mission  capable 
rate  of  70%,  a  fleet  of  53  aircraft  would  have  28  aircraft  available  for  training.  Each 
scheduled  aircraft  executes  an  average  of  12.94%  more  sorties  than  the  current  fleet  of  65 
aircraft.  When  the  fleet  mission  capable  rate  increases  to  80%,  there  are  once  again  3 1 
aircraft  available  for  sorties  and  the  daily  sorties  are  0.5%  lower  than  the  current  fleet. 
The  minimal  decrease  in  sorties  per  aircraft  noted  in  Table  5  is  especially  noteworthy 
because  it  validates  the  T-X  Utilization  Rate  Model.  The  T-X  Utilization  Rate  Model 
was  previously  developed  by  the  Air  Force  to  determine  the  projected  fleet  size  of  350 
aircraft  (Michalec,  2013).  At  a  fleet  mission  capable  rate  of  80  percent,  53  aircraft  could 
do  the  same  work  as  the  current  65  without  requiring  a  significant  change  in  workload. 

The  data  in  tables  three  through  five  support  the  current  plan  to  procure  350 
replacement  aircraft.  Even  though  the  proposed  fleet  is  less  than  80  percent  the  size  of 
the  current  fleet,  it  would  still  be  able  to  maintain  student  pilot  throughput. 

Minimizing  Resources 

The  next  phase  of  this  study  explores  the  minimum  quantities  of  aircraft  and 
GBTSs  required  to  maintain  on-time  student  pilot  throughput.  The  model  ran  the 
simulation  multiple  times  with  incrementally  lower  numbers  of  resources  until  on-time 
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student  throughput  dropped  below  120  students.  The  table  displays  scenarios  with  the 
lowest  number  of  resources  that  can  still  support  120-student  throughput.  Test  scenarios 
included  the  extreme  objective  and  threshold  levels  at  the  three  mission  capable  rates 
described  by  tables  three  through  five  above.  The  results  are  listed  in  Table  6  below. 


Table  6.  Minimize  Resources 


Input 

Results 

Break 

% 

# 

# 

% 

# 

# 

# 

Rate, 

Ao 

Acft 

G 

Acft-Ao 

MC  Acft 

Sorties/ 

Sorties/ 

Fix 

w/53 

B 

on  Fly 

Scheduled 

Ao 

Rate, 

Acft 

T 

Schedule 

Acft/ 

GBTS/ 

Turn 

Time, 

GBTS 

S 

Day 

Day 

-Ao 

T 

64.68 

38 

7 

50.06 

14.27 

4.47 

2.85 

±0.36 

±0.1 

±0.06 

±0.02 

0 

64.68 

35 

8 

46.82 

12.29 

5.16 

2.20 

±0.33 

±0.09 

±0.07 

±0.01 

T 

69.93 

34 

10 

52.53 

13.39 

4.74 

1.99 

±0.29 

±0.07 

±0.05 

±0.01 

0 

69.93 

32 

7 

50.35 

12.08 

5.27 

2.52 

±0.31 

±0.08 

±0.06 

±0.02 

T 

80.00 

30 

7 

64.27 

14.46 

4.37 

2.83 

±0.27 

±0.06 

±0.05 

±0.02 

0 

80.00 

28 

7 

62.66 

13.16 

4.82 

2.52 

±0.32 

±0.07 

±0.06 

±0.02 

T=  Threshold  level 

95%  Confidence  Intervals 

0=Objective  level 

The  data  in  Table  6  indicates  that  although  the  previous  tests  did  not  show 
significant  variability  when  fleet  size  is  constant,  the  delta  between  threshold  and 
objective  levels  has  significant  impact  on  potential  fleet  size.  At  first  look,  it  appears  that 
dozens  of  aircraft  could  be  cut  without  affecting  the  Air  Force’s  ability  to  maintain  the 
same  graduate  throughput.  The  number  of  sorties  each  aircraft  would  have  to  fly, 
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however,  indicates  that  these  levels  are  not  feasible.  Each  aircraft  on  the  flying  schedule 
would  have  to  fly  at  least  four  sorties  a  day  and  in  some  of  the  scenarios  more  than  five. 
There  are  only  enough  hours  in  the  day  for  an  aircraft  to  fly  five  complete  sorties.  This 
means  that  every  mission  capable  aircraft  would  have  to  be  on  the  flying  schedule  every 
day  in  order  to  achieve  the  mission.  In  accordance  with  current  maintenance  practices, 
and  as  stated  earlier  in  this  study,  25%  of  the  mission  capable  aircraft  are  not  placed  on  a 
given  day’s  flying  schedule.  This  set  of  reserved  aircraft  ensures  that  the  breaks  that 
occur  today  do  not  adversely  affect  the  follow  days’  schedules. 

The  data  also  shows  that  with  a  given  scheduled  maintenance  plan,  the  mission 
capable  rate  drops  when  the  fleet  shrinks.  This  insight  is  often  forgotten,  as  it  is 
sometimes  assumed  that  mission  capable  rates  are  scalable  inputs  dependent  on 
individual  aircraft  rather  than  flying  hours  or  number  of  sorties  flown. 

Addition  analysis  helped  maintain  realistic  flying  schedules  as  well  as  hold  25% 
of  the  mission  capable  aircraft  in  reserve  for  the  following  day.  To  understand  the 
minimum  number  of  aircraft,  one  must  understand  the  minimum  number  of  scheduled 
aircraft  separate  from  reserved  or  broken  aircraft  in  the  fleet.  To  do  this,  the  model  ran 
objective  and  threshold  scenarios  that  excluded  scheduled  maintenance.  Aircraft  break 
rates  were  still  factored  in  because  breaks  happen  during  the  day  and  reduce  the  available 
number  of  scheduled  aircraft.  To  exclude  nonscheduled  maintenance  assumes  that 
broken  aircraft  are  replaced  that  same  day  by  reserved  aircraft.  No  consideration  is  given 
to  such  replacement  since  it  is  not  a  desired  plan  of  action.  The  result  are  shown  below  in 
Table  7. 
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Table  7.  Minimized  Scheduled  Resources 


Input 

Results 

Break 

% 

# 

# 

% 

# 

# 

# 

Rate, 

Ao 

Acft 

G 

Acft-Ao 

MC  Acft 

Sorties/ 

Sorties/ 

Fix 

w/53 

B 

on  Fly 

Scheduled 

Ao 

Rate, 

Acft 

T 

Schedule 

Acft/ 

GBTS/ 

Turn 

Time, 

GBTS 

S 

Day 

Day 

-Ao 

T 

— 

20 

7 

94.33 

20 

3.36 

2.84 

±0.09 

±0.03 

±0.02 

0 

— 

17 

7 

94.46 

17 

3.64 

2.53 

±0.08 

±0.03 

±0.02 

T=  Threshold  level 

95%  Confidence  Intervals 

0=Objective  level 

The  data  in  Table  7  shows  the  minimum  number  of  exclusively  scheduled  aircraft 
can  support  the  syllabus  flow  of  students.  Unlike  the  scenarios  in  Table  6,  these  two 
scenarios  simulate  a  situation  where  reserved  aircraft  cannot  fly  in  the  model.  The 
number  of  daily  sorties  each  aircraft  flies  are  still  high,  but  the  scenarios  are  feasible. 

Hypothesis  testing  validated  that  the  combinations  of  aircraft  and  GBTS  in 
Table  7  are  statistically  significant.  Figure  8  below  depicts  the  results  of  testing  the  null 
hypothesis  that  the  average  student  throughput  equals  120  students  per  year.  With  an 
alpha  value  of  0.05,  the  tests  fail  to  reject  the  null  hypothesis  with  the  combinations  listed 
in  Table  7.  Figure  8  shows  that  if  the  number  of  aircraft  are  reduced,  from  Thresh20  to 
Thresh  19  and  from  Obj  17  to  Obj  16,  our  results  reject  the  null  hypotheses.  The  data 
shows  that  with  19  aircraft  in  a  threshold  scenario,  or  16  aircraft  in  an  objective  scenario, 
the  goal  of  120  annual  pilot  graduates  is  not  sustainable  at  the  95%  confidence  level.  A 
similar  evaluation  applied  to  reducing  GBTS  also  verified  the  results  of  Table  7  and 
indicated  that  less  than  seven  GBTS  cannot  support  required  annual  student  throughput. 
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®Thresh20 


>  Quantiles 


A\*  Summary  Statistics 

Mean  119.88 

StdDev  0.7182746 

Std  Err  Mean  0.1015794 
Upper  95%  Mean  120.08413 
Lower  95%  Mean  1 1 9.67587 
N  50 

Test  Mean 

Hypothesized  Value  120 
Actual  Estimate  119.88 
DF  49 

StdDev  0.71827 

tTest 

Test  Statistic  -1.1813 
Prob  >  |t|  0.2432 

Prob>t  0.8784 

Prob  <t  0.1216 


A\*  Thresh19 


[  Quantiles 


a  ▼  Summary  Statistics 

Mean  119.08 

StdDev  1.5364948 

Std  Err  Mean  0.2172932 
Upper  95%  Mean  119.51667 
Lower  95%  Mean  118.64333 
N  50 

A  ▼  Test  Mean 
Hypothesized  Value  120 
Actual  Estimate  119.08 
DF  49 

StdDev  1.53649 

tTest 

Test  Statistic  -4.2339 
Prob  >  |t|  0.0001* 

Prob>t  0.9999 

Prob<t  <0001* 


A  ▼  Obj17 


[>  Quantiles 


A  ▼  Summary  Statistics 

Mean  119.94 

StdDev  0.3136357 

Std  Err  Mean  0.0443548 
Upper  95%  Mean  120.02913 
Lower  95%  Mean  11 9.85087 
N  50 

A  ▼  Test  Mean 
Hypothesized  Value  120 
Actual  Estimate  119.94 
DF  49 

StdDev  0.31364 

tTest 

Test  Statistic  -1.3527 
Prob  >  |t|  0.1824 

Prob  >t  0.9088 

Prob  <t  0.0912 


A  ▼  Obj16 


[>  Quantiles 


a  ▼  Summary  Statistics 

Mean  117.06 

Std  Dev  4.3066631 

Std  Err  Mean  0.6090541 
Upper  95%  Mean  118.28394 
Lower  95%  Mean  115.83606 
N  50 

A  ▼  Test  Mean 
Hypothesized  Value  120 
Actual  Estimate  117.06 
DF  49 

Std  Dev  4.30666 

tTest 

Test  Statistic  -4.8272 
Prob  >  |t|  <0001* 

Prob  >t  1.0000 

Prob<t  <0001* 


Figure  8.  Statistical  Significance  of  Minimized  Scheduled  Resources 


Using  the  infonnation  from  Table  7,  the  model  reevaluates  the  scenarios  given  in 
Table  6.  The  results  are  subject  to  the  added  constraints  that  objective  scenarios  cannot 
have  any  fewer  than  17  aircraft  scheduled  and  threshold  scenarios  cannot  have  any  fewer 
than  20  aircraft  scheduled.  Table  8  below  gives  the  results  of  the  reevaluated  minimum 
resources. 

Flypothesis  testing  for  Table  8  results  use  the  null  hypothesis  of  number  of 
scheduled  aircraft  equals  17  or  20  for  objective  and  threshold  scenarios  respectively.  The 
results  shown  in  Figure  9  confirm  Table  8  scenarios  are  statistically  significant,  at  a  = 
0.05,  and  meet  annual  student  throughput  and  scheduled  resource  constraints. 
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Table  8.  Minimized  Fleet  Scenarios 


Input 

Results 

Break 

% 

# 

# 

% 

Minimum 

# 

# 

Rate, 

Ao 

Acft 

G 

Acft-Ao 

# 

Sorties/ 

Sorties/ 

Fix 

w/53 

B 

MC  Acft 

Scheduled 

Ao 

Rate, 

Acft 

T 

on  Fly 

Acft/ 

GBTS/ 

Turn 

S 

Schedule 

Day 

Day 

Time, 

GBTS 

-Ao 

T 

64.68 

46 

7 

59.33 

20 

3.16 

2.84 

±0.26 

±0.02 

±0.02 

0 

64.68 

42 

7 

55.93 

17 

3.73 

2.52 

±0.28 

±0.02 

±0.02 

T 

69.93 

43 

7 

62.7 

20 

3.18 

2.85 

±0.28 

±0.02 

±0.02 

0 

69.93 

39 

7 

59.65 

17 

3.73 

2.52 

±0.24 

±0.02 

±0.01 

T 

80.00 

38 

7 

72.16 

20 

3.15 

2.82 

±0.16 

±0.02 

±0.01 

0 

80.00 

33 

7 

68.6 

17 

3.71 

2.51 

±0.2 

±0.02 

±0.02 

T=  Threshold  level 

95%  Confidence  Intervals 

0=0bjective  level 

-  T64  68 


Quantiles 


Summary  Statistics 

Uean  30  468505 

Sid  Dev  0  3120643 

Sid  Err  Mean  0044319 
OPPfr  95%  Mt»n  2055736 
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N  50 
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Prob»t  «000V 
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a  -  Summary  Statistics 
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Upper  95%  Mean  17  706534 
Lower  05%  Mean  17530122 
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Ptob-I  *0001* 

Prob  « I  10000 


A  »  TB9.93 
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i 
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r 

- 
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Mean  20220686 
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a  *  Test  Mean 
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Actual  Csamale  202207 
Of  40 
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Prob'H  '0001* 
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Prob  «l  10000 


A  -  069  TO 


a  -  Summary  Statistics 
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a  -  Summary  Statistics 
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Sid  Dev  0  1607077 
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upper  95%  Mean  20  509905 
Lower  05%  Mean  20  518550 
N  50 

a  »  Test  Maan 


-080 
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Mean  16  977015 

SM  Dev  0  1701013 
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M  50 

a  •  Test  Mean 
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OF  49 
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treat 
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Figure  9.  Statistical  Significance  of  Minimized  Fleet  Scenarios 
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The  data  from  Table  8  gives  a  practical  look  at  the  minimum  number  of  aircraft 
and  GBTSs  required  to  support  the  current  syllabus  with  consideration  to  family  of 
system  attributes.  Even  though  these  fleet  sizes  are  technically  possible,  they  are  not 
necessarily  desirable.  Decision  makers  can  see  from  the  number  of  daily  sorties  data  that 
reduced  fleet  sizes  require  more  aircraft  turns  than  current  operations  usually  support.  In 
order  to  implement  the  fleet  scenarios  in  Table  8,  scheduled  aircraft  would  have  to  fly 
57%  to  86%  more  than  they  currently  do  with  a  fleet  of  65  aircraft. 
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V.  Conclusions  and  Recommendations 


Conclusions  of  Research 

The  study  demonstrates  that  an  analysis  of  aircraft  FoS  attributes  in  conjunction 
with  student  pilot  syllabus  requirements  identifies  potential  procurement  savings  for  the 
Air  Force.  This  study  supports  the  current  plan  to  purchase  350  aircraft.  The  study  also 
suggests  that  a  different  fleet  size  may  be  cheaper  and  just  as  effective.  If  a  proposed 
trainer  FoS  performs  above  threshold  perfonnance  levels,  it  may  be  more  economical  to 
purchase  fewer  of  those  aircraft.  Likewise,  it  may  be  more  economical  to  purchase  more 
aircraft  that  are  less  capable  but  sufficient.  The  model  developed  in  this  study  allows 
decision  makers  to  determine  the  right  size  of  the  fleet  based  on  proposed  options  for  the 
T-X  FoS  and  pilot  training  throughput  requirements.  After  pilot  training  syllabi  are 
updated  to  include  bridge  course  requirements,  the  model  in  this  study  can  be  update  to 
provide  a  better  estimate  of  fleet  requirements. 

Significance  of  Research 

This  research  combined  previous  studies  that  focused  on  student  throughput  and 
aircraft  maintenance  modeling  into  a  single  integrated  model.  With  a  combination  of 
both  elements,  the  model  provides  decision  makers  with  a  mission  oriented  approach  to 
requirements  planning.  In  the  case  of  Air  Force  pilot  student  training,  the  mission  is  to 
graduate  quality  students.  The  mission  is  not  just  to  move  students  through  the  system  as 
fast  as  possible  nor  is  it  to  ensure  that  aircraft  generations  reach  a  certain  number.  The 
combined  look  provided  additional  insight  into  the  minimum  number  of  resources 
required  to  complete  the  mission. 
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With  mission  accomplishment  as  the  key  goal,  this  study  showed  that  aircraft 
availability  rates  could  descend  much  lower  than  current  thresholds.  This  suggests  that 
although  Ao  metrics  can  be  useful  for  improvement  goals  or  measures  of  efficiency,  they 
are  not  a  good  measure  of  mission  effectiveness  in  a  training  environment.  The  Ao 
should  not  be  discounted,  but  it  should  also  not  be  used  as  a  single  point  of  evaluation. 

This  study  also  highlighted  issues  with  MTBM,  Break  Rate,  and  Fix  Rate  criteria. 
One  or  all  of  these  criteria  need  to  be  relaxed  in  order  for  the  threshold  level  of  each  to  be 
concurrently  feasible  with  a  realistic  scheduled  maintenance  plan.  This  insight  is 
beneficial  not  only  to  improve  the  metrics  themselves,  but  also  to  highlight  a  potential 
problem  with  procurement  requirements.  The  scheduled  maintenance  plan  determines 
the  vast  majority  of  the  Ao  metric.  The  Ao  metric  is  evaluated  with  only  20,000  fleet 
hours,  a  value  that  the  fleet  will  hit  after  only  a  few  months.  With  these  circumstances, 
there  is  a  risk  that  vendors  will  forego  some  scheduled  maintenance  requirements  in  the 
short  run  to  improve  the  Ao  metric.  This  could  result  in  aircraft  that  require  more 
maintenance  in  the  long  run  and  dramatically  lower  operational  availability  after  20,000 
fleet  hours  have  been  flown.  It  may  be  beneficial  for  the  Air  Force  to  lengthen  the  fleet 
hours  to  encompass  a  year  or  two  of  fleet  hours  in  order  to  capture  all  scheduled 
maintenance  requirements. 

In  addition  to  providing  a  tool  to  right  size  a  potential  family  of  systems,  the  tool 
can  also  help  decision  makers  educate  others  on  the  impact  of  reducing  fleet  structures. 

If  the  fleet  size  is  reduced  after  a  contract  has  already  been  awarded  to  a  vendor,  this  tool 
will  show  what  the  Air  Force  can  expect  from  the  future  fleet’s  performance.  This  tool 
shows  that  arbitrarily  shrinking  a  fleet  size  after  maintenance  requirements  have  been  set 
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significantly  impacts  the  operational  ability  of  aircraft.  Mission  capable  rates  are  based 
on  fleets,  not  individual  aircraft.  The  tool  helps  to  highlight  this  truth  by  showing  the 
decrease  in  availability  and  the  increase  in  the  number  of  sorties  each  remaining  aircraft 
will  assume  in  order  to  maintain  the  same  level  of  service. 

Recommendations  for  Future  Research 

This  study  provided  an  introduction  between  combining  student  flow  with  aircraft 
availability  models.  Both  sides  of  the  issue  could  be  further  developed  to  generate  more 
insights.  Research  could  be  expanded  to  include  constraints  on  available  number  of 
instructors  as  well  as  inclusion  of  multilevel  maintenance  requirements.  Any  further 
development  of  the  model  must  continue  to  focus  on  key  aspects  to  keep  analysis 
relatively  simple. 

Another  area  of  future  research  could  be  a  cost  analysis  of  the  decided  upon  fleet 
size.  Potential  savings  are  already  identified  by  reducing  resources,  but  the  model  does 
not  address  the  costs  associated  with  those  savings.  Cost  should  be  evaluated  specifically 
in  additional  maintenance  requirements  from  higher  rates  of  aircraft  turns  per  day,  as  well 
as  manpower  requirements  to  support  a  multitum  flying  schedule. 
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